Artificial Neural Architecture Research Articles

Large scale integrated IGZO crossbar memristor array based artificial neural architecture for scalable in-memory computing

Neuromorphic systems based on memristor arrays have not only addressed the von Neumann bottleneck issue but have also enabled the development of computing applications with high accuracy. In this study, an artificial neural architecture based on a 10 × 10 IGZO memristor array is presented to emulate synaptic dynamics for performing artificial intelligence (AI) computing with high recognition accuracy rate. The large area 10 × 10 IGZO memristor array was fabricated using the photolithography method, resulting in stable and reliable memory operations. The bipolar switching at −2 V–2.5 V, endurance of 500 cycles, retention of >104 s, and uniform Vset/Vreset operation of 100 devices were achieved by modulating the oxygen vacancy in the IGZO film. The emulation of electric synaptic dynamics was also observed, including potentiation-depression, multilevel long-term memory (LTM), and multilevel short-term memory (STM), revealing highly linear and stable synaptic functions at different modulated pulse settings. Additionally, electrical modeling (HSPICE) with vector-matrix measurements and simulation of various artificial neural network (ANN) algorithms, such as convolution neural network (CNN) and spiking neural network (SNN), were performed, demonstrating a linear increase in current accumulation with high recognition rates of 99.33 % and 86.46 %, respectively. This work provides a novel approach for overcoming the von Neumann bottleneck issue and emulating synaptic dynamics in various neural networks with high accuracy.

Exploring the benefits of images with frequency visual content in predicting human ocular scanpaths using Artificial Neural Networks

We present a study of an artificial neural architecture that predict human ocular scanpaths while they are free-viewing different images types. This analysis is made by comparing different metrics that encompass scanpath patterns, these metrics aim to measure spatial and temporal errors; such as the MSE, ScanMatch, cross-correlogram peaks, and MultiMatch. Our methodology begin by choosing one architecture and training different parametric models per subject and image type, this allows to adjust the models to each person and a given set of images. We find out that there is a clear difference in prediction when people free-view images with high visual content (high-frequency contents) and low visual content (no-frequency contents). The input features selected for predicting the scanpath are saliency maps calculated from foveated images together with the past of the ocular scanpath of subjects, modeled by our architecture called FovSOS-FSD (Foveated Saliency and Ocular Scanpath with Feature Selection and Direct Prediction).The results of this study could be used to improve the design of gaze-controlled interfaces, virtual reality, as well as to better understand how humans visually explore their surroundings and pave a way to make future research.

A Logically Reversible Double Feynman Gate with Molecular Engineered Bacteria Arranged in an Artificial Neural Network-Type Architecture.

Reversible logic gates are the key components of reversible computing that map inputs and outputs in a certain one-to-one pattern so that the output signals can reveal the pattern of the input signals. One of the main research foci of reversible computing is the implementation of basic reversible gates by various modalities. Though true thermodynamic reversibility cannot be attained within living cells, the high energy efficiency of biological reactions inspires the implementation of reversible computation in living cells. The implementation of synthetic genetic circuits is mostly based on conventional irreversible computing, and the implementation of logical reversibility in living cells is rare. Here, we constructed a 3-input-3-output synthetic genetic reversible double Feynman logic gate with a population of genetically engineered E. coli cells. Instead of following hierarchical electronic design principles, we adapted the concept of artificial neural networks (ANN) and built a single-layer artificial network-type architecture with five different engineered bacteria, named bactoneurons. We used three extracellular chemicals as input signals and the expression of three fluorescence proteins as the output signals. The cellular devices, which combine the input chemical signals linearly and pass them through a nonlinear activation function and represent specific bactoneurons, were built by designing and creating small synthetic genetic networks inside E. coli. The weights of each of the inputs and biases of individual bactoneurons in the bacterial ANN were adjusted by optimizing the synthetic genetic networks. When arranging the five bactoneurons through an ANN-type architecture, the system generated a double Feynman gate function at the population level. To our knowledge, this is the first reversible double Feynman gate realization with living cells. This work may have significance in development of biocomputer technology, reversible computation, ANN wetware, and synthetic biology.

Rethinking Motivation of Deep Neural Architectures

Nowadays, deep neural architectures have acquired great achievements in many domains, such as image processing and natural language processing. In this paper, we hope to provide new perspectives for the future exploration of novel artificial neural architectures via reviewing the proposal and development of existing architectures. We first roughly divide the influence domain of intrinsic motivations on some common deep neural architectures into three categories: information processing, information transmission and learning strategy. Furthermore, to illustrate how deep neural architectures are motivated and developed, motivation and architecture details of three deep neural networks, namely convolutional neural network (CNN), recurrent neural network (RNN) and generative adversarial network (GAN), are introduced respectively. Moreover, the evolution of these neural architectures are also elaborated in this paper. At last, this review is concluded and several promising research topics about deep neural architectures in the future are discussed.

Estimation of the Perceived Time of Presence of Sources in Urban Acoustic Environments Using Deep Learning Techniques

The impact of urban sound on human beings has often been studied from a negative point of view (noise pollution). In the two last decades, the interest of studying its positive impact has been revealed with the soundscape approach (resourcing spaces). The literature shows that the recognition of sources plays a great role in the way humans are affected by sound environments. There is thus a need for characterizing urban acoustic environments not only with sound pressure measurements but also with source-specific attributes such as their perceived time of presence, dominance or volume. This paper demonstrates, on a controlled dataset, that machine learning techniques based on state of the art neural architectures can predict the perceived time of presence of several sound sources at a sufficient accuracy. To validate this assertion, a corpus of simulated sound scenes is first designed. Perceptual attributes corresponding to those stimuli are gathered through a listening experiment. From the contributions of the individual sound sources available for the simulated corpus, a physical indicator approximating the perceived time of presence of sources is computed and used to train and evaluate a multi-label source detection model. This model predicts the presence of simultaneously active sources from fast third octave spectra, allowing the estimation of perceptual attributes such as pleasantness in urban sound environments at a sufficient degree of precision.

Complete vector quantization of feedforward neural networks

Deep neural networks are widely used to solve several difficult machine learning tasks due to their impressive performance on standard benchmarking datasets. Most of the state of the art neural architectures contain a staggering amount of parameters and have many layers. Thus, they are computationally intensive and memory demanding models. This fact prohibits their deployment in devices with limited computational resources such as smartphones and unmanned vehicles. Recently, a growing interest in developing methods that can compress and accelerate these networks emerged. In this paper, we propose the use of complete vector quantization for neural model compression and acceleration. More specifically, we show that it is possible to use product quantization with common subdictionaries to quantize both the parameters and the activations of the neural network without compromising significantly the network accuracy. The proposed method removes the need for multiplications in order to compute the neural preactivations and provides opportunities for acceleration using lookup tables.

Artificial Synapse Emulated by Charge Trapping‐Based Resistive Switching Device

AbstractThe traditional Von Neumann architecture‐based computers are considered to be inadequate in the coming artificial intelligence era due to increasing computation complexity and rising power consumption. Neuromorphic computing may be the key role to emulate the human brain functions and eliminate the Von Neumann bottleneck. As a basic unit in the nervous system, a synapse is responsible for transmitting information between neurons. Resistive random access memory (RRAM) is able to imitate the synaptic functions because of its tunable resistive switching behavior. Here, an artificial synapse based on solution processed polyvinylpyrrolidone (PVPy)–Au nanoparticle (NP) hybrid is fabricated, various synaptic functions including paired‐pulse facilitation (PPF), posttetanic potentiation (PTP), transformation from short‐term plasticity (STP) to long‐term plasticity (LTP) and learning‐forgetting‐relearning process are emulated, making the polymer–metal NPs hybrid system valuable candidates for the design of novel artificial neural architectures.

Episodic memory transfer for multi-task reinforcement learning

Abstract Episodic memory plays important role in animal behavior. It allows to reuse general skills for solution of specific tasks in changing environment. This beneficial feature of biological cognitive systems is still not incorporated successfully in an artificial neural architectures. In this paper we propose a neural architecture with shared episodic memory for multi-task reinforcement learning (SEM-PAAC). This architecture extends Parallel Advantage Actor Critic (PAAC) with two recurrent sub-networks for separate tracking of environment and task states. The first subnetwork store episodic memory and the second one allows task specific execution of policy. Experiments in the Taxi domain demonstrated that SEM-PAAC has the same performance as PAAC when subtasks are solved separately. On the other hand when subtasks are solved jointly for completing full Taxi task SEM-PAAC is significantly better due to reuse of episodic memory. Proposed architecture also successfully learned to predict task completion. This is a step towards more autonomous agents for multitask problems.

High-order fuzzy-neuro-entropy integration-based expert system for time series forecasting

In this study, a new fuzzy time series (FTS) forecasting model is introduced. The proposed model tries to deal with four major issues associated with the FTS modeling approach, viz., determination of the lengths of intervals, determination of fuzzy logical relations (FLRs), inclusion of repeated FLRs, and defuzzification operation. For the determination of effective lengths of intervals, “frequency-based discretization” approach is proposed, which partitions the time series data set into various lengths. For establishment of the FLRs and their better representation, an artificial neural network-based architecture is developed. Another novel idea is contributed to the article by considering repeated FLRs during forecasting. Last but not the least, this study introduces a new “entropy-based defuzzification” technique for the defuzzification operation. The model is verified and validated with real-world data sets. Various comparison studies and performance evaluations over the data sets signify the effectiveness and efficiency of the model.

A conceptual framework of artifical neural network

Artificial Neural Networks are non-linear structures based on the function of human brain. They are highly flexible function approximators for any data. In this paper, we discuss about Artificial Neural Network, its components, architecture and types of neural networks based on different types learning methodology. We have also elaborated on applications of ANNs in various fields. An ANN is also being constructed taking Sum function as propagation function and Sigmod as activation function for a simple neuron.

A Self-Organized Artificial Neural Network Architecture for Sensory Integration with Applications to Letter-Phoneme Integration

The multimodal self-organizing network (MMSON), an artificial neural network architecture carrying out sensory integration, is presented here. The architecture is designed using neurophysiological findings and imaging studies that pertain to sensory integration and consists of interconnected lattices of artificial neurons. In this artificial neural architecture, the degree of recognition of stimuli, that is, the perceived reliability of stimuli in the various subnetworks, is included in the computation. The MMSON's behavior is compared to aspects of brain function that deal with sensory integration. According to human behavioral studies, integration of signals from sensory receptors of different modalities enhances perception of objects and events and also reduces time to detection. In neocortex, integration takes place in bimodal and multimodal association areas and result, not only in feedback-mediated enhanced unimodal perception and shortened reaction time, but also in robust bimodal or multimodal percepts. Simulation data from the presented artificial neural network architecture show that it replicates these important psychological and neuroscientific characteristics of sensory integration.

An advanced combination strategy for multi-classifiers employed in large multi-class problem domains

Traditional artificial neural architectures possess limited ability to address the scale problem exhibited by a large number of distinct pattern classes and limited training data. To address these problems, this paper explores a novel advanced encoding scheme, which reduces both memory demand and execution time, and provides improved performance. The novel advanced encoding scheme known as the engine encoding, have been implemented in a multi-classifier, which combines the scaled probabilities, configuration information, and the discriminating abilities of the associated component classifiers. The problems of overloading and saturation experienced by traditional networks are solved by training the base classifiers on differing sub-sets of the required pattern classes and allowing the combiner classifier to derive a solution. Current Multi-classifier Systems are easily biased when trained on one class more often than another class, when patterns representing a class are very large compared to the rest, or when the multi-classifier depends on a certain fixed order of arrangement of pattern classes. A unique statistical arrangement method is hereby presented which aims to solve the bias problem. This statistical arrangement method also enhances independence of component classifiers. The system is demonstrated on the exemplar of fingerprint identification and utilizes a Weightless Neural System called the Enhanced Probabilistic Convergent Neural Network ( EPCN) in a Multi-classifier System .

Prune-Able Fuzzy ART Neural Architecture for Robot Map Learning and Navigation in Dynamic Environments

Mobile robots must be able to build their own maps to navigate in unknown worlds. Expanding a previously proposed method based on the fuzzy ART neural architecture (FARTNA), this paper introduces a new online method for learning maps of unknown dynamic worlds. For this purpose the new Prune-able fuzzy adaptive resonance theory neural architecture (PAFARTNA) is introduced. It extends the FARTNA self-organizing neural network with novel mechanisms that provide important dynamic adaptation capabilities. Relevant PAFARTNA properties are formulated and demonstrated. A method is proposed for the perception of object removals, and then integrated with PAFARTNA. The proposed methods are integrated into a navigation architecture. With the new navigation architecture the mobile robot is able to navigate in changing worlds, and a degree of optimality is maintained, associated to a shortest path planning approach implemented in real-time over the underlying global world model. Experimental results obtained with a Nomad 200 robot are presented demonstrating the feasibility and effectiveness of the proposed methods.

Learning sensor-based navigation of a real mobile robot in unknown worlds

In this paper, we address the problem of navigating an autonomous mobile robot in an unknown indoor environment. The parti-game multiresolution learning approach is applied for simultaneous and cooperative construction of a world model, and learning to navigate through an obstacle-free path from a starting position to a known goal region. The paper introduces a new approach, based on the application of the fuzzy ART neural architecture, for on-line map building from actual sensor data. This method is then integrated, as a complement, on the parti-game world model, allowing the system to make a more efficient use of collected sensor information. Then, a predictive on-line trajectory filtering method, is introduced in the learning approach. Instead of having a mechanical device moving to search the world, the idea is to have the system analyzing trajectories in a predictive mode, by taking advantage of the improved world model. The real robot will only move to try trajectories that have been predicted to be successful, allowing lower exploration costs. This results in an overall improved new method for goal-oriented navigation. It is assumed that the robot knows its own current world location-a simple dead-reckoning method is used for localization in our experiments. It is also assumed that the robot is able to perform sensor-based obstacle detection (not avoidance) and straight-line motions. Results of experiments with a real Nomad 200 mobile robot are presented, demonstrating the effectiveness of the discussed methods.

Local cluster neural net: Architecture, training and applications

This paper describes the structure, training and computational abilities of the local cluster (LC) artificial neural net architecture. LC nets are a special class of multilayer perceptrons that use sigmoid functions to generate localised functions. LC nets train as fast as radial basis functions nets and are more general. They are well suited for both, multi-dimensional function approximation and discrete classification. The LC net is the result of our search for a widely applicable neural net architecture suitable for low-cost hardware realisation. The LC net seem particularly well suited for analog VLSI realisation of small-size, low-power, fully parallel neural net chip for real time control applications.

An adaptive algorithm for speech analysis and synthesis.

In this paper, an artificial neural net architecture for adaptive speech analysis and synthesis is proposed. Based on discussing the signal analysis procedure with mathematical method, the simulation experiment is performed firstly on a typical signal (a square wave). Compared with Fast Fourier Transform, some important characteristics of the Neural Net Signal Analyzer-Synthesizer (NNSAS) are drawn from the experiment results. Then, NNSAS is used for speech signal processing, which results in a new approach to speech analysis and synthesis. In terms of the auditory masking effect, only some major components of connection strength vector are used to produce synthetic speech. Under the high compressive rate conditions, NNSAS can still produce synthetic speech with high articulation.

Dynamic Cell Structure Learns Perfectly Topology Preserving Map

Dynamic cell structures (DCS) represent a family of artificial neural architectures suited both for unsupervised and supervised learning. They belong to the recently (Martinetz 1994) introduced class of topology representing networks (TRN) that build perfectly topology preserving feature maps. DCS employ a modified Kohonen learning rule in conjunction with competitive Hebbian learning. The Kohonen type learning rule serves to adjust the synaptic weight vectors while Hebbian learning establishes a dynamic lateral connection structure between the units reflecting the topology of the feature manifold. In case of supervised learning, i.e., function approximation, each neural unit implements a radial basis function, and an additional layer of linear output units adjusts according to a delta-rule. DCS is the first RBF-based approximation scheme attempting to concurrently learn and utilize a perfectly topology preserving map for improved performance. Simulations on a selection of CMU-Benchmarks indicate that the DCS idea applied to the growing cell structure algorithm (Fritzke 1993c) leads to an efficient and elegant algorithm that can beat conventional models on similar tasks.

An artificial neural emulator for an odour sensor array

In this paper we describe the development of a stand-alone microprocessor -based instrument which can classify intelligently the signals from an array of odour-sensitive sensors. Data from the odour sensor array are initially used to train a neural net on a PC using a software program and then sent down an RS-232C interface to the artifical neural emulator (ANE). The ANE uses multiplexer and ADC chips to condition the array output, followed by a Motorola 6803-based digital processing stage that contains in its RAM or ROM, an artificial neural architecture, neuronal weights and an operating instruction set. The processor computes the output of a neural network, classifies this against a knowledge base of odours and displays the result on a two-line LCD display. The ANE has also been fused with an array four commercial gas sensors (Figaro Engineering Inc., Japan), a temperature IC and a capacitive humidity sensor to produce a stand-alone intelligent odour-sensing system.

Software Design for Neurocontrol Code

This paper discusses software engineering aspects of neurocontrol algorithm design. Cutting edge algorithm development often outstrips the capability of routinely available software packages. Hence, it is difficult to perform adequate algorithm validation and verification (algorithm V & V). The basic building blocks of a software neurocontroller (data sampling, state prediction and control selection) are developed. The underlying artificial neural architecture used here is the CMAC, although much of the discussion is fairly generic. Code fragments are provided that illus trate implementation details for the CMAC architecture, the sampling of the data stream, local con troller estimates and so forth.

Investigation of artificial neural system architectures using actors: [formula omitted] Science Applications International Corporation, 10260 Campus Point Drive, San Diego, CA 92121 USA

Investigation of artificial neural system architectures using actors: [formula omitted] Science Applications International Corporation, 10260 Campus Point Drive, San Diego, CA 92121 USA